Skip to content

HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach…#2762

Merged
xiaoyuyao merged 4 commits intoapache:trunkfrom
xiaoyuyao:HADOOP-17578
Mar 17, 2021
Merged

HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach…#2762
xiaoyuyao merged 4 commits intoapache:trunkfrom
xiaoyuyao:HADOOP-17578

Conversation

@xiaoyuyao
Copy link
Contributor

@xiaoyuyao xiaoyuyao commented Mar 11, 2021

https://issues.apache.org/jira/browse/HADOOP-17578

Additional Debug Log for troubleshooting TokenCache related issues.

@xiaoyuyao xiaoyuyao closed this Mar 12, 2021
@xiaoyuyao xiaoyuyao reopened this Mar 12, 2021
cxorm
cxorm approved these changes Mar 15, 2021
View reviewed changes
Copy link
Member

@cxorm cxorm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @xiaoyuyao for the work.

Overall looks great, I'm +1.
Do you mind adding a empty-commit to trigger the CI-checks again ?

sadanand48
sadanand48 approved these changes Mar 15, 2021
View reviewed changes
Copy link
Contributor

@sadanand48 sadanand48 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 . The changes look good to me. Please resolve the failing CI checks.

@hadoop-yetus
Copy link

hadoop-yetus commented Mar 15, 2021

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 35s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 32m 33s trunk passed
+1 💚 compile 20m 54s trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 compile 18m 0s trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 checkstyle 1m 6s trunk passed
+1 💚 mvnsite 1m 33s trunk passed
+1 💚 javadoc 1m 5s trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 1m 35s trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 spotbugs 2m 22s trunk passed
+1 💚 shadedclient 15m 45s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 52s the patch passed
-1 ❌ compile 8m 4s /patch-compile-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt root in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
-1 ❌ javac 8m 4s /patch-compile-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt root in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
-1 ❌ compile 7m 13s /patch-compile-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt root in the patch failed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.
-1 ❌ javac 7m 13s /patch-compile-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt root in the patch failed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 47s the patch passed
+1 💚 mvnsite 1m 12s the patch passed
-1 ❌ javadoc 0m 46s /patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt hadoop-common in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
+1 💚 javadoc 1m 20s the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 spotbugs 2m 16s the patch passed
-1 ❌ shadedclient 5m 52s patch has errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 17m 25s hadoop-common in the patch passed.
+1 💚 asflicense 0m 32s The patch does not generate ASF License warnings.
141m 1s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/2/artifact/out/Dockerfile
GITHUB PR #2762
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 4ad9cd0410d5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / a02fc561ec3aadf9f303aebcfa2fe754db180972
Default Java Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/2/testReport/
Max. process+thread count 2003 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

hadoop-yetus commented Mar 16, 2021

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 20s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 35m 25s trunk passed
+1 💚 compile 23m 42s trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 compile 19m 29s trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 checkstyle 1m 0s trunk passed
+1 💚 mvnsite 1m 30s trunk passed
+1 💚 javadoc 1m 0s trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 1m 32s trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 spotbugs 2m 20s trunk passed
+1 💚 shadedclient 18m 13s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 55s the patch passed
+1 💚 compile 21m 51s the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javac 21m 51s the patch passed
+1 💚 compile 18m 58s the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 javac 18m 58s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 57s /results-checkstyle-hadoop-common-project_hadoop-common.txt hadoop-common-project/hadoop-common: The patch generated 1 new + 71 unchanged - 0 fixed = 72 total (was 71)
+1 💚 mvnsite 1m 27s the patch passed
-1 ❌ javadoc 1m 0s /patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt hadoop-common in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
+1 💚 javadoc 1m 33s the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 spotbugs 2m 32s the patch passed
+1 💚 shadedclient 18m 30s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 17m 5s hadoop-common in the patch passed.
+1 💚 asflicense 0m 48s The patch does not generate ASF License warnings.
191m 7s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/3/artifact/out/Dockerfile
GITHUB PR #2762
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 2a22a533a829 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 54b0ae4
Default Java Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/3/testReport/
Max. process+thread count 2144 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/3/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

hadoop-yetus commented Mar 17, 2021

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 59s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+1 💚 mvninstall 35m 22s trunk passed
+1 💚 compile 22m 38s trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 compile 19m 12s trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 checkstyle 1m 0s trunk passed
+1 💚 mvnsite 1m 31s trunk passed
+1 💚 javadoc 0m 58s trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javadoc 1m 32s trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 spotbugs 2m 20s trunk passed
+1 💚 shadedclient 18m 2s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 53s the patch passed
+1 💚 compile 22m 2s the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚 javac 22m 2s the patch passed
+1 💚 compile 19m 15s the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 javac 19m 15s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 1m 3s the patch passed
+1 💚 mvnsite 1m 25s the patch passed
-1 ❌ javadoc 0m 57s /patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt hadoop-common in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
+1 💚 javadoc 1m 32s the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚 spotbugs 2m 31s the patch passed
+1 💚 shadedclient 18m 15s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 17m 14s hadoop-common in the patch passed.
+1 💚 asflicense 0m 49s The patch does not generate ASF License warnings.
189m 3s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/4/artifact/out/Dockerfile
GITHUB PR #2762
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 206807b5ac97 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 6655d22
Default Java Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/4/testReport/
Max. process+thread count 2533 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/4/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@xiaoyuyao
Copy link
Contributor Author

xiaoyuyao commented Mar 17, 2021

Java doc issue is unrelated. Thanks @cxorm for the review. I'll merge the PR shortly.

@xiaoyuyao xiaoyuyao merged commit b503de2 into apache:trunk Mar 17, 2021
@aajisaka
Copy link
Member

aajisaka commented Mar 18, 2021

Java doc issue is unrelated.

[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2762/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java:1923: error: exception not thrown: java.io.IOException
[ERROR]    * @throws IOException
[ERROR]              ^

throws IOException has been removed by the patch and it is related.

@xiaoyuyao @cxorm
Would you please fix this?

@xiaoyuyao xiaoyuyao deleted the HADOOP-17578 branch March 19, 2021 20:22
@steveloughran
Copy link
Contributor

steveloughran commented Mar 22, 2021

yeah, we do need a fix for this....

@xiaoyuyao
Copy link
Contributor Author

xiaoyuyao commented Mar 22, 2021

Thanks @steveloughran and @aajisaka. I will open a new ticket for the javadoc fix.

@xiaoyuyao
Copy link
Contributor Author

xiaoyuyao commented Mar 23, 2021

#2802 is opened to fix the javadoc issue.

kiran-maturi pushed a commit to kiran-maturi/hadoop that referenced this pull request Nov 24, 2021
@xiaoyuyao
HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… (
7f6c37f 
apache#2762)
jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023
@xiaoyuyao
HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… (
3102f84 
apache#2762)

(cherry picked from commit b503de2)
Change-Id: I9957dd1b80720966829dd49cff133223b90829cf
jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023
Revert "HADOOP-17578. Improve UGI debug log to help troubleshooting T…
144f739 
…okenCach… (apache#2762)"

This reverts commit 3102f84.

Reason for revert: <INSERT REASONING HERE>

Change-Id: I159965c018f89ba7e495f0467034ce1fc2ea8bdd
jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023
@xiaoyuyao
HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… (
71e1232 
apache#2762)

(cherry picked from commit b503de2)
Change-Id: I25ced4aebca6092d4f08c1d9d5e355419bd36b38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@cxorm cxorm cxorm approved these changes

+1 more reviewer

@sadanand48 sadanand48 sadanand48 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@xiaoyuyao @hadoop-yetus @aajisaka @steveloughran @sadanand48 @cxorm